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INTRODUCTION 


Substance misuse and related behavioral health problems 
such as injury, addiction, and overdose are pressing 
personal and public health concerns. To successfully 
address these problems in states, tribes, jurisdictions, and 
communities, prevention planners need information about 
the effectiveness of available programs and practices. They 
also need to know how to determine which options have 


the greatest potential to work well in their unique settings 


and how to proceed if no viable options are available. 


To help meet these needs, the Substance Abuse and Mental Health Services Administration (SAMHSA) has 
developed this guidance document, Selecting Best-Fit Programs and Practices: Guidance for Substance 
Misuse Prevention Practitioners. This resource places the selection of programs and practices within the 


broader context of evidence-based prevention. Specifically, it explores the following: 
e The value of embedding program and practice selection in a strategic planning process 
e Where to find information on programs and practices and how to choose among them 


e Tips for adopting, adapting, and innovating programs and practices and for supporting their successful 


implementation and continual improvement at the local level 


Supporting materials at the end of this resource provide additional information on these topics. 


BEFORE THE SEARCH: LAYING A FOUNDATION THROUGH STRATEGIC PLANNING 


Prevention researchers and practitioners have developed ; 
What Are Programs and Practices? 


many programs and practices that are capable of producing Abe caraaeceneimrederermned 


positive changes for individuals, families, and communities Pircturedmandiceerdinsredmenrminers 


struggling with substance use-related problems. Yet in order practice is a type of approach, technique, 


to work, these programs and practices must be carefully or strategy. 


selected and implemented and then continually improved 
; . . ; A program can incorporate different 
over time. This can be ensured through strategic planning— ? : ; ; 
practices, and guidance for implementing 
a systematic process for setting priorities and goals and for me ; 
a specific practice can be developed and 
determining both how to achieve them and how to know Aa. 
distributed as a program. 


they are being achieved. 
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SAMHSA’S STRATEGIC PREVENTION FRAMEWORK 
SAMHSA’s Strategic Prevention Framework (SPF) is a proven strategic planning model comprising five steps: 


e Step 1. Assessment involves gathering and using data to identify a priority problem, factors influencing 


this problem, and resources and readiness to address it. 


e Step 2. Capacity involves building resources and readiness to address the priority problem and its 


associated factors. 


e Step 3. Planning involves developing a comprehensive plan that details prevention priorities, programs 


and practices selected to address them, and anticipated outcomes. 


e Step 4. Implementation involves moving the prevention plan into action by fine-tuning selected 


programs and practices and delivering them as intended. 


e Step 5. Evaluation involves examining how programs and practices are working and using lessons 


learned to improve them and the plan overall. 


These five steps are typically presented in circular form (Figure 1) 


Figure 1. 
because the SPF process is iterative and dynamic; planners often cycle SAMHSA’s STRATEGIC 
back to earlier steps and engage in multiple steps simultaneously. For PREVENTION FRAMEWORK 


example, they may need to adjust their comprehensive plan if 
ongoing assessment efforts reveal shifting prevention priorities or 
build additional capacity to support a specific program or practice 
once it is underway. In addition, the overall SPF process is guided by 


two principles that should be integrated into each step: “eezultaral 


e Cultural competence, which is the ability of an individual 


or an organization to interact effectively with members of 


diverse population groups 


e Sustainability, which is the capacity of a community to produce and maintain positive prevention 


outcomes after initial funding ends and over time 


Together, these principles dictate that all prevention efforts must be informed by and responsive to the unique 
cultures of those involved, and that individuals, families, and communities should continue to reap the health- 
related benefits of prevention efforts over time. Successful completion of each step and integration of both 
principles require the active participation of and collaboration among diverse community stakeholders. These 
individuals and institutions may change as a prevention initiative evolves, but the need for prevention partners 


will remain constant. 
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THE SPF APPROACH TO SELECTING PROGRAMS AND PRACTICES 


Successful movement through the early steps of the SPF generates the information that planners need to set 
prevention priorities, identify intended outcomes, and build a logic model to inform the selection of programs 


and practices. Each of these functions is described below. 
Setting Prevention Priorities 


Every community struggles with multiple substance use-related problems, but no community can address 
them all—at least not all at once. Setting clear priorities requires understanding which problems are most 
important for a community to address first, and which problems a community is most capable of changing. By 
engaging in a thorough assessment of local prevention needs and capacity, and in a collaborative prioritization 
process, planners can identify their community’s priority problem. This begins to focus their prevention 


initiative. 


But a community cannot address a substance use-related 
ee ee kth hth coe Multiple Factors, Multiple Levels 
roblem directly; it must work through the underlyin ; : ; 
P ¥ 6 oe Risk factors are associated with an 


factors that influence this problem. For this reason, fgereaeediicelingediiatelpesontal 


planners also need to identify the priority risk and protective experience a problem. Protective factors 


factors they intend to address in order to influence their arevacsociatad with a decreased likelihood: 


priority problem. This requires an understanding of which Both types of factors operate at different 
risk and protective factors are present and most urgent at levels of a person’s experience. 
the local level, and which of these factors the community is 
. 3 SOCIETY 
in a strong position to change. 
COMMUNITY 
For example, an assessment of prevention needs and 
, a ee ee RELATIONSHIP 
capacity may reveal a community’s priority problem to be 
the nonmedical use of prescription drugs (NMUPD) among INDIVIDUAL 


youth. This assessment may further reveal that the best way 
to address this problem is by reducing two priority risk factors: perception among youth that prescription 
drugs are safer than other drugs and youth access to prescription drugs; and by strengthening two priority 


protective factors: positive familial bonds and parental disapproval of prescription drug misuse. 
Identifying Anticipated Outcomes 


By setting clear prevention priorities, planners begin to articulate what their community intends to accomplish 
through its prevention efforts. To inform the selection and evaluation of programs and practices to address 


these priorities, planners must take this a step further and specify what their community intends to 
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accomplish. This is done by identifying anticipated outcomes. There are two types of outcomes: short-term 


outcomes and long-term outcomes. 


e Short-term outcomes are the immediate effects of programs and practices, such as changes in 


people’s knowledge, skills, and/or access to substances. 


e Long-term outcomes are the ultimate effects of programs and practices after they have been in place 


for a while, such as changes in substance use behaviors and related health problems. 


From Short- to Long-Term Outcomes 
Long-term outcomes, which can take several years to produce, usually result from an accumulation of short- 


term outcomes. A community can feel confident it is moving in the right direction if it sees anticipated 


changes in the risk and protective factors associated with its priority problem. 


SHORT-TERM OUTCOMES LONG-TERM OUTCOMES 


Changes in risk and Changes in behaviors 
protective factors and related problems 


For example, prevention planners in a community addressing NUUPD among youth might determine that a 20 
percent increase in parents reporting disapproval of prescription drug misuse and a 10 percent decrease in 


youth reporting NMUPD-related behaviors are realistic short- and long-term outcomes, respectively. 
Developing a Logic Model to Inform Selection 


Priority problems and factors reflect where a community is; anticipated outcomes indicate where it wants to 
go. To capture these key decisions, and inform the selection of programs and practices to move a community 
in the right direction, planners can begin to build a logic model: a graphic planning tool that makes explicit the 
rationale for selecting programs and practices by presenting the relationship between problems, associated 


factors, selected programs and practices, and anticipated outcomes (Figure 2). 


Figure 2. LOGIC MODEL FOR PREVENTION 


Short- 
and 


Prevention 
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When complete, a logic model for prevention reveals a community’s plan for addressing its priority substance 
use-related problem. Some communities may only have the capacity to support a single prevention program or 
practice; these communities can be strategic about selecting the one that is likely to have the greatest impact. 
But when and where possible, there is added value in taking a comprehensive approach to prevention. This 
type of plan includes multiple programs and practices designed to address different risk and protective factors 


in different community settings—including homes, schools, health care facilities, neighborhoods, and more. 


THE SEARCH: FINDING AND SELECTING PREVENTION PROGRAMS AND PRACTICES 


The best candidates for inclusion in a community’s comprehensive prevention plan are programs and practices 


with strong conceptual fit, practical fit, and evidence of effectiveness. 


e Conceptual fit is the degree to which a program or practice is a good match for the job that needs to 
be done; for example, a saw is a good match for the job of cutting a piece of wood—better than a 


hammer or screwdriver. 


e Practical fit is the degree to which a program or practice is a good match for the people involved and 
the community overall; for example, a handsaw is a good match for someone who wants to cut wood 


but who cannot afford or comfortably operate a power saw. 


e Evidence of effectiveness is the proof that a program or practice can (or cannot) do the job that needs 
to be done; for example, watching someone use a handsaw to cut through wood is evidence of that 


specific saw’s effectiveness. 


Figure 3 presents a process for identifying best-fit programs and practices—that is, those with strong 


conceptual fit, practical fit, and evidence of effectiveness. Each step in this process is described below. 


Figure 3. IDENTIFYING BEST-FIT PREVENTION PROGRAMS AND PRACTICES 


© FIND OPTIONS Best-Fit Options 


Strong Fit & Strong 
© CONSIDER STRENGTH OF EVIDENCE | Evidence of Effectiveness 


© CONSIDER FIT 
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FINDING OPTIONS 


Communities want to establish prevention programs and practices that work. To maximize the chances of this 
happening, planners should limit their initial search to those for which there is some evidence of potential to 
produce positive prevention outcomes. By focusing on these evidence-based programs and practices (EBPPs), 
planners will have concrete information to support their decision-making process. They can find this 
information in systematic reviews of the effectiveness of available EBPPs and, as needed, individual evaluation 
studies of EBPPs. 


Systematic Reviews 


Many groups with expertise in and commitment to evidence-based prevention conduct systematic reviews of 
the effectiveness of available programs and practices. These reviews are excellent sources of information for 

planners searching for EBPPs that may be a good fit for their communities. Findings from such reviews can be 
found in searchable online databases and publications from federal agencies, other prevention and public 


health organizations (e.g., national nonprofits, university-based research centers), and peer-reviewed journals. 
Individual Studies 


Expertly conducted systematic reviews offer prevention planners a valuable snapshot of information across 
multiple evaluation studies of EBPPs. However, some planners may not find an option with strong conceptual 
and practical fit for their communities in these reviews. If this happens, planners may want to look closely at 
findings from evaluation studies of individual prevention programs and practices. This information can be 
found in peer-reviewed journals as well as in reports written by those involved in the implementation and 
evaluation of programs and practices at the local level (e.g., evaluation reports for funding agencies and to 


support prevention planning, doctoral theses). 
CONSIDERING FIT 


No matter how much evidence of effectiveness exists for an EBPP, it will only be appropriate for a community 


if it is actually the right fit. There are two types of fit: conceptual and practical. 
Conceptual Fit 


To determine conceptual fit, or how well-suited a program or practice is for doing a specific job, planners can 


look closely at their community’s logic model for prevention. An EBPP with strong conceptual fit is one that: 


e Directly addresses the community’s priority substance use-related problem as well as one or more 


priority risk and protective factors associated with that problem 


e Has been shown to produce positive outcomes among members of the community’s focus 


population(s) 
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For example, a parenting program that raises awareness of the 
dangers of prescription drug misuse would have strong conceptual 
fit for a community that identified parental disapproval of 
prescription drug misuse as a priority protective factor for 
addressing NMUPD among youth. In contrast, a responsible 
beverage server training for alcohol retailers would have weak 
conceptual fit for this community as it addresses neither NUUPD 


among youth nor parental attitudes toward substance misuse. 


Practical Fit 


To determine practical fit, or how well-suited a program or practice is to a community overall, prevention 
planners can look closely at their community’s capacity—that is, its readiness to support new prevention 
efforts and the resources it has in place, or that could be leveraged, to do so. An EBPP with strong practical fit 


is one that: 


e ls supported by key prevention stakeholders and the broader community 
e Is feasible for the organization/community to implement well 


e Complements existing prevention efforts in the community 


For example, a parenting program to reduce NMUPD among youth would have strong practical fit in a 
community in which many residents and leaders recognize that NMUPD is a problem; local parents are ready 
and willing to address it; the program includes content and strategies that are appropriate for these parents; 
and the program is both affordable for the implementing organization and aligned with its mission and other 
activities. In contrast, a parenting program would have weak practical fit if key stakeholders are unaware that 


NMUPD is even a problem and the implementing organization is unwilling or unable to accommodate it. 


Does It Fit? 

An EBPP with strong conceptual fit will fit neatly into a community’s 
logic model for prevention. One with strong practical fit will fit neatly 
into the implementing organization and broader community. 

There will always be work involved in getting a new program or practice 
up and running. But if it has strong fit, this work should meet with little 
resistance—like placing a puzzle piece right where it belongs. 


SELECTING BEST-FIT PROGRAMS AND PRACTICES: GUIDANCE FOR SUBSTANCE MISUSE PREVENTION PRACTITIONERS 


What If Nothing Fits? 


Prevention planners who are unable to find any EBPPs with strong conceptual fit and strong practical fit may 


want to adjust their search criteria and/or process. For example: 


e /f there are no EBPPs that directly address their priority problem and factors, planners may want to 
consider options that address the same priority factors (e.g., parental disapproval) for a different 


priority problem (e.g., underage drinking rather than NMUPD). 


e If there are no EBPPs that address their priorities among members of their focus population(s), 
planners may want to consider options that address a different priority problem among their focus 


population(s) (e.g., alcohol misuse rather than opioid misuse among Native American adults). 


e If there are no EBPPs that their community is willing and able to support at this time, planners may 


want to work on building capacity for prevention prior to continuing their search. 


CONSIDERING STRENGTH OF EVIDENCE 


As mentioned earlier, an EBPP with strong conceptual fit is one with 
evidence that it directly addresses local priorities and can produce 
intended outcomes. But not all evidence is created equal. Considering 
the strength of an EBPP’s evidence of effectiveness involves closely 


examining how the evidence was gathered and determining how much 


confidence it deserves. 


Evidence of effectiveness falls along a continuum, from strong to weak. The stronger the evidence, the more 
confidence it deserves. Strong evidence that an EBPP is, or is not, effective comes from strong evaluation 


studies; the more scientifically rigorous, numerous, and varied the studies, the more compelling the evidence. 


The following criteria are often used to assess the strength of evaluation evidence: 


e Research design describes the approach and structure of the research study. Its purpose is to ensure 
the study yields information that can answer the research question both meaningfully and 
unambiguously. As scientific rigor of the research design increases, so too does confidence in the 


information that is gathered and shared. 


e Internal validity is the degree to which a program or practice can be considered responsible for 
producing the outcomes measured in an evaluation study. As scientific rigor of the research design 


increases, so too does confidence in the internal validity of the results. 
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e Independent replication is the degree to 
which a program or practice found to 
produce results with one set of participants 
consistently produces the same results when 
rigorously implemented and evaluated by 
independent practitioners or researchers 


with other similar sets of participants. 


e External and ecological validity is the degree 
to which a program or practice found to 
produce results with one set of participants 
consistently produces the same results when 
rigorously implemented and evaluated with 
other different sets of participants (external 


validity) and under real-world conditions 


What Does Rigor Look Like? 
Different research designs possess different 
degrees of scientific rigor. An experimental 
design is typically considered the most 
rigorous. In this design, study participants 
are randomly assigned to an intervention 
(i.e., program or practice) group or toa 
control group. Results from both groups are 
compared, both before and after the 
intervention. This ability to compare groups 
enables the researchers to isolate and 
identify any effects produced by the 
intervention and rule out other possible 


explanations for these effects. 


(ecological validity). 


For example, a program for suburban White parents would have very strong evidence of effectiveness for this 
population if rigorous independent evaluations with members of this population under real-world conditions 
all demonstrate consistently positive results. However, this program would only have strong evidence of 
effectiveness for members of a different population—such as rural Latino parents—if it achieved similar results 


for this population through a similarly rigorous set of evaluation studies. 


Prevention planners can use the criteria presented above to understand common categorizations of EBPPs in 


resources such as federal registries and reports. For example: 


e EBPPs with the strongest and most favorable evidence of effectiveness are typically referred to as well- 


supported, model, or exemplary. 

e EBPPs with weaker yet still favorable evidence of effectiveness are typically referred to as supported, 
promising, or emerging. 

e  EBPPs with insufficient empirical evidence to draw meaningful conclusions about their effectiveness 


are typically referred to as inconclusive or undetermined. 


e _EBPPs with unfavorable evidence of effectiveness are typically referred to as unsupported (strong 
evidence that they do not produce desired outcomes) or harmful (any evidence, regardless of scientific 


rigor, that they produce negative outcomes). 
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Figure 4. COMMON CATEGORIES BASED ON STRENGTH OF EVIDENCE SUPPORTING EFFECTIVENESS 


PROGRAM OR PRACTICE EFFECTIVENESS 
Strong 


Well- 
Supported, 
Model, or 


evidence 
Exemplary 


of negative 
outcomes 


Supported 


Promising 


Emerging 


STRENGTH OF EVIDENCE 


Inconclusive 
or 
Weak Undetermined 


Adapted from Puddy, & Wilkins, 2011, p. 8 


EBPPs that appear on the far left side of the figure above and also have strong fit should rise to the top of a 
planner’s lists of candidates for inclusion in a community’s comprehensive prevention plan. Those on the far 
right side, which either do not work or can be harmful, should be dismissed outright. Options in the middle 
categories with strong fit should be carefully considered, particularly when no alternatives are available with 
stronger evidence of effectiveness. An EBPP with limited evidence of effectiveness may work well in and for a 
community—and a community can help build its evidence base by evaluating it and sharing the results with 


others in the prevention field. 


AFTER THE SEARCH: MAXIMIZING THE POTENTIAL FOR SUCCESS 


When searching for an EBPP, prevention planners may find a 
best-fit option, a viable but imperfect option, or no viable option 
at all. Each of these outcomes can set their community on a 


different path: adoption, adaptation, or innovation. 


READY TO GO: ADOPTING A BEST-FIT OPTION 


As described in the previous section, a best-fit EBPP is one with 


strong evidence of effectiveness in addressing a community’s 
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unique prevention priorities, as well as strong fit for the community overall. A community that finds an EBPP 
with these ideal characteristics can usually adopt it outright and implement it with fidelity—that is, with strict 
adherence to its original design. This course of action increases the community’s chances of reproducing the 


positive outcomes this EBPP produced elsewhere. 


There are many important ways to promote implementation fidelity and effectiveness. These are described 


later in this section, in Maximizing Potential: Building Supports for Successful Implementation. 


Fidelity and Effectiveness 

According to Durlak and DuPre (2008), “The difference favoring programs with apparently better as opposed 
to poorer implementation is profound, and has resulted in mean effect sizes [differences between groups] 
that are two to three times higher, and, under ideal circumstances, may be up to 12 times higher” (p. 330). 


These findings underscore the importance of implementation fidelity in prevention. 


FROM GOOD TO BEST: ADAPTING A VIABLE BUT IMPERFECT OPTION 


While implementation fidelity is strongly associated with effectiveness, some departures from an EBPP’s 
original design and delivery are inevitable. According to Janevic et al. (2016), “The need to modify evidence- 
based interventions when they are implemented in new practice settings is somewhere between common and 
universal” (p. 1). Durlak and DuPre (2008) acknowledge this as well; in addition to emphasizing the importance 
of fidelity, they state that it is unrealistic to expect perfect implementation in real-world settings and that 
positive outcomes can be achieved even at implementation levels well below 100 percent. In fact, some 
changes, or adaptations, can even improve the potential of an EBPP to produce positive outcomes—in 


particular, those adaptations that are carefully planned and executed. 


Planned adaptations can help improve an EBPP’s potential effectiveness by addressing recognized deficiencies 
related to fit. For example, if prevention planners find an EBPP that was designed to address their community’s 


priority problem among members of a different 
? 
focus population, they might consider ways to What Core Components? 
. . . ; Making meaningful changes while retaining core 
improve its cultural fit—that is, the relevance of 
. : components seems like an ideal way to balance the 
the language, attitudes, beliefs, values, and 
. : . need for real-world fit and high fidelity—but core 
experiences reflected in the EBPP’s design. 
. . . components are not always readily apparent. If you 
When planning adaptations of an EBPP, it is 
: : — are unsure about the core components of your 
important to strive to retain its core 
: a selected EBPP, seek guidance from the original 
components—that is, the specific elements that 
: . : developer(s) and/or others who have used and 
are required and responsible for producing 
. . Das aie evaluated it. 
positive outcomes. The following guidelines can 


a1 
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help communities make adaptations that retain core components and boost, rather than compromise, 


effectiveness: 


e Preserve the setting. It may be unrealistic or impossible to make an EBPP designed for one setting 


(e.g., schools) appropriate for a different setting (e.g., health clinics). 


e Maintain the dosage, including the number, length, and spacing of sessions. Sufficient participant 


exposure may be essential for effectiveness. 


e Add new content if the need for content changes arise, rather than subtract existing content. This will 


prevent the removal of core content. 


e Make any design or delivery changes with intention and care. Work closely with the original 
developers (if implementing a program), members of and leaders from your community’s focus 
population, and other experts in prevention and program evaluation to execute adaptations— 


including the addition of new content. 


For example, in one American Indian community looking to offer a parenting program to prevent youth 
substance use, community members worked with university researchers to culturally adapt an evidence-based 
program that was originally developed for Latino parents. They selected this program because it reflected 
many of the risk and protective factors prioritized by 
all involved, including supporting youth in different 
cultural environments. Together, they incorporated 
American Indian cultural values, worldviews on 
parenting, and family challenges specific to Native 
experience as well as cultural elements like 
storytelling that are common across diverse tribal 
communities. Participants in the adapted program 
reported increases in their parenting skills and Native 


cultural identity and decreases in negative behaviors 


among their children (Kulis, Ayers, & Baker, 2015). 
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WHY WAIT? INNOVATE! DEVELOPING A NEW PROGRAM OR PRACTICE 


Sometimes the most thorough search for EBPPs fails to 
generate a single best-fit option or even a solid starting 
point for adaptation. This tends to occur when looking 
for programs or practices that address priority problems 
with emerging rather than extensive evidence—for 
example, opioid overdose as compared with underage 
drinking. This may also be the case when looking for 
programs or practices designed for and/or evaluated 


with traditionally underserved focus populations—for 


example, people with disabilities. 


In the absence of viable EBPPs, prevention planners might choose to expand or delay their search. Or they may 
choose to innovate—that is, develop a new or “home-grown” program or practice. Through innovation, 
communities can address their unique prevention needs in a uniquely appropriate manner. By documenting 
and evaluating these innovations, they can also help the prevention field evolve and benefit others in similar 


situations. 


Innovation, like any other prevention-related decision and effort, should be grounded in evidence. While a 
new program or practice cannot be deemed evidence-based until after it has been evaluated for effectiveness, 
it can and should be evidence-informed. Valuable sources of expertise and insights that can support an 


evidence-informed innovation process include relevant: 


e Research: This may include studies of how other groups and communities have tried to address 
problems and/or serve populations of interest, including what did and did not work well—and why. It 
may also include more general theories and studies of behavior change; the starting point for many 
effective programs and practices is a broad but substantiated theory of change, and no program or 


practice can be considered evidence-based without this foundation. 


e Practice: Prevention planners can look to practices that are currently being used in diverse community 
settings to address problems and/or serve populations of interest. These may include culture-based 
practices, which are informed and guided by the social structures, values, and beliefs of specific 
cultural groups. Many effective prevention programs and practices have grown out of real-world 
approaches to promoting health and well-being that communities agree are working well—that is, 


practice-based evidence. 
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e EBPPs: Even when no EBPPs are the right fit for a community, prevention planners can still benefit 
from looking closely at those with some relevance to their community’s priority problem and/or focus 
population(s). Doing so can help them understand the wide range of options, explore varied best 
practices, and crystalize their thinking about what will—or will not—work well for their community. 


e Associations: Prevention planners can also consult . on 
Virtual Communities 


with groups at the local, state, regional, and national prevenucnielannericancloconeuian tn 


levels dedicated to advancing best practices in ae 
fellow planners working in other 


addressing substance use-related problems and/or ae : ; 
communities to develop innovative 


supporting specific cultural populations. These prevention and public health interventions. 


groups include professional associations focused on : shee : 
Virtual communities of practice, such as 


specific prevention strategies, such as the National NNEDehare dom ThOINGROGalNGauortG 


Association of Drug Court Professionals, and/or Eliminate Disparities in Behavioral Health, 


specific disciplines, such as the National Association a verweluaile esperiumhtse tar 


of Addiction Treatment Providers. These groups also : , : 
information-sharing and support. 


include diverse cultural centers and associations. 


For example, members of a university-tribal partnership in the Pacific Northwest developed an innovation to 
promote cultural identity and prevent substance misuse among tribal youth based on a cultural practice 
known as the Canoe Journey. This intertribal tradition, which has included nearly 100 tribes in a given year, 
includes the formation of Canoe Families within each tribe comprised of youth, their families and extended 
families, and other tribal and nontribal community members. Each Canoe Family meets throughout the year, 
participating in drug-free cultural events and fundraising efforts to support the annual Canoe Journey. Many 
tribal youth and Canoe Family participants refer to the Canoe Journey as their most highly valued cultural 


best practice for prevention (Hawkins, Cummins, & Marlatt, 2004). 


Adoption, Adaptation, and Innovation: Distinct Yet Overlapping Paths 


It is important to keep in mind that adoption, adaptation, and ADOPTION 
innovation are not mutually exclusive. For example, minor 

adaptations are common when adopting an EBPP; adaptations can 
turn an EBPP into something new and innovative; and developers ADAPTATION INNOVATION 


of a new program or practice want their innovation to be adopted. 
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MAXIMIZING POTENTIAL: BUILDING SUPPORTS FOR SUCCESSFUL IMPLEMENTATION 


The most carefully selected or crafted prevention effort will only work well if it is implemented, from the very 
start and over time, with the same degree of care. So whether adopting, adapting, or innovating a new 
program or practice, prevention planners and communities will need to invest in its implementation and 
continual improvement to ensure its success. Specifically, they can consider important factors and take 


decisive actions in each of the following areas: 


e Provider selection: The providers responsible for implementing a new program or practice should be 
committed to its delivery, qualified and confident in their ability to implement it, a good cultural match 


for the focus population, and willing to learn—before and throughout its implementation. 


e Provider preparation and support: Essential learning opportunities for providers include pre- and in- 
service trainings to promote the knowledge and skills needed to implement the program or practice as 


intended, as well as ongoing consultation and coaching to provide on-the-job support and assistance. 


e Process and outcome evaluation: By closely monitoring the delivery of a program or practice, 
communities can make sure it is being implemented as intended and improved as needed. By 
assessing program or practice outcomes, communities can determine whether it is working as 
intended and worth investing in and continuing over time. By sharing this information, communities 
can help build the evidence base for programs and practices—thereby contributing to the prevention 


literature and giving other communities more valuable information to support prevention planning. 


e Organizational leadership and prevention champions: 
Strong, dedicated leaders can foster an organizational 
culture supportive of change, including the use of new 
prevention programs and practices; help keep all involved 
coordinated and energized; and proactively remove on- 
site implementation barriers. These leaders, along with 
other prevention champions, can also work with systems 
beyond the implementation site to ensure the 


continuation of policies, funding, and other supports 


conducive to implementation and continual improvement. 


e Implementation guidance: According to the Centers for Disease Control and Prevention (as cited in 
Puddy, & Wilkins, 2011, p. 19), “Implementation guidance includes any and all services and/or 
materials that aid in the implementation of a prevention strategy in a different setting, including but 


not limited to: training, coaching, technical assistance, support materials, organization/system change 
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consultation, and manuals/guides.” Such guidance from program or practice developers, and/or others 
with relevant skills and experience, can support a community’s efforts in each of the areas presented 


above. 


Each of these implementation supports is necessary and essential, but it is through a combination of these 
supports that a community can truly maximize the potential for individual program or practice success—just as 
it is through a combination of prevention programs and practices that a community can truly move the needle 


on its priority substance use-related problem. 


No matter how well any single program or practice fits or is implemented 
within a given organization and community, its power is strengthened 
exponentially by being part of a strategically planned and comprehensive 
prevention initiative. In fact, many of the anticipated outcomes identified 
in a community’s logic model for prevention, in particular long-term 
changes in substance use behaviors and related consequences, rely on the 
comprehensive plan as a whole. At the same time, each program and 


practice within that plan must be carefully selected, implemented, and 


continually improved if it—or the plan as a whole—is to be successful. 
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APPENDIX 1: PROGRAM AND PRACTICE REVIEW CHECKLIST 


Best-fit prevention programs and practices are those with strong conceptual fit, practical fit, and evidence of 
effectiveness. This checklist includes some key considerations in each of these areas. While not exhaustive, this 
checklist can help prevention planners create useful snapshots of program and practice viability for their 
communities. Planners can complete one checklist for each program or practice they review, then compare 


strength of fit and evidence of effectiveness across multiple programs and practices. 


Name of program/practice: 


Source(s) of information used to complete this checklist: 


CONCEPTUAL FIT 


O) Alignment with priority problem: This program/practice is designed to address the specific substance 


use-related problem our community has prioritized. 


O) Alignment with priority factor(s): This program/practice directly addresses one or more of the specific 


risk or protective factors our community has prioritized. 


O) Alignment with focus population(s): This program/practice is designed for use with our community’s 


focus population(s) for prevention efforts. 


C1 Evidence of outcome(s) of interest: This program/practice has been formally evaluated to determine 


its impact on our community’s anticipated short- and/or long-term prevention outcome(s). 


PRACTICAL FIT 


C1 Implementation guidance: Program/practice materials (e.g., facilitator guide) and/or services (e.g., 
training and technical assistance) are available that detail its content, specify its requirements, and can 


aid in its implementation. Please note: This guidance can help you consider the items below. 


CL) Support of key stakeholders: This program/practice is likely to be supported by those it will serve, 
those who will be responsible for its implementation, and others with relevant decision-making power 


in our community. 


CL) Support of the broader community: This program/practice is likely to be supported by others in our 
community, including those who may not be directly involved in or affected by its implementation but 


are invested in our priority problem and/or focus population(s). 
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PRACTICAL FIT (continued) 


O) Feasibility of implementation: It is likely that the implementation site will have sufficient capacity to 
meet this program’s/practice’s requirements for use—including funds for materials and training, time 


and space, and access to qualified staff and evaluators as well as intended participants. 


C1 Synergy with other prevention efforts: This program/practice aligns well with the mission of the 
implementation site and supports/enhances other prevention efforts at this site and in the broader 


community. 


EVIDENCE OF EFFECTIVENESS 


If your information sources include a systematic review of multiple evaluation studies, complete this item. If 


not, skip to the next set of items. 


Systematic review designation: Based on the strength of evidence supporting its effectiveness in producing 


intended outcomes, the following best describes the reviewers’ categorization of this program/practice: 


C Well-supported, model, or exemplary (strongest and most favorable evidence of effectiveness) 

CO Supported, promising, or emerging (weaker yet still favorable evidence of effectiveness) 

C1 inconclusive or undetermined (insufficient empirical evidence to draw meaningful conclusions about 
effectiveness) 

1 Unsupported (strong evidence that it does not produce intended outcomes) 


O Harmful (any evidence that it produces negative outcomes) 
If your information sources include one or more evaluation studies, complete the following items. 


C1 implementation fidelity: This program/practice was implemented with strict adherence to its original 
design. Please note: This can increase confidence that the program/practice evaluated is the one you 


are considering. 


C Study design: This program/practice was evaluated using a scientifically rigorous research design. 
Please note: This can increase confidence that the program/practice itself was responsible for 


producing outcomes of interest (i.e., internal validity). 


C1 study outcomes: This program/practice produced positive outcomes similar to our community’s 


anticipated short- and/or long-term prevention outcomes. 


C1 Study participants: The study participants are similar to our community’s focus population(s). 
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EVIDENCE OF EFFECTIVENESS (continued) 


C1 Independent replication: This program/practice has produced consistently positive outcomes across 


multiple rigorous studies with similar sets of participants. 


1 external validity: This program/practice has produced consistently positive outcomes across multiple 


rigorous studies with different sets of participants. 


1 Ecological validity: This program/practice has produced positive outcomes under real-world conditions 


(e.g., staff turnover, cancelled sessions). 


SUMMARY OF FINDINGS AND NEXT STEPS 


Based on this checklist, this program/practice appears to have strong: 
C1 Conceptual fit 
1 Practical fit 
1 Evidence of effectiveness 


If a meaningful conclusion in each of these areas is not possible at this time, some ways to learn more 


about this program/practice include the following: 
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APPENDIX 2: SUPPORTING MATERIALS 


The following resources are intended to help prevention planners dig deeper into some of the important topics 


and concepts introduced in this guidance document: 


Collaboration Across the Strategic Prevention Framework 
Criteria for Setting Prevention Priorities 

Key Features of Risk and Protective Factors 

Logic Models for Prevention 

Characteristics of a Comprehensive Approach 


Finding Evidence-Based Programs and Practices 


Now PWN bP 


Types of Evaluation 
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1. COLLABORATION ACROSS THE STRATEGIC PREVENTION FRAMEWORK 


No single individual or agency can provide the resources or reach needed to effectively address a community’s 


prevention needs. This is why collaboration is such an integral part of SAMHSA’s Strategic Prevention 


Framework (SPF). To influence complex social problems, practitioners must tap the skills and resources of a 


broad spectrum of community stakeholders throughout the prevention planning process. Following are 


examples of opportunities for collaboration at each step of the SPF: 


SPF Step 


Step 1: Assessment: 
Communities use 
data to explore local 
prevention needs 
and capacity and 
identify a priority 


problem 


Opportunities for Collaboration 


Obtain and manage data on local substance 
use problems and related behaviors 
Identify a priority problem 

Obtain and manage data on the risk and 
protective factors associated with and local 
capacity to address the priority problem 
Share and receive feedback on assessment 


findings 


Example 


Engage evaluators and/or graduate 
students from a local university to 
help collect, analyze, and interpret 


assessment data 


Step 2: Capacity: 
Communities build 
local resources and 
readiness to address 


the priority problem 


Develop and/or strengthen a prevention 
team (e.g., add new members to fill gaps in 
expertise and increase access to resources) 
Raise community awareness of the priority 
problem 

Promote community support for and 


participation in prevention efforts 


Work with the editor of a local 
newspaper to highlight community 
prevention efforts in a monthly 


news column 


Step 3: Planning: 
Communities 
develop a 
comprehensive plan 
for addressing the 


priority problem 


Identify specific risk and protective factors to 
address in order to influence the priority 
problem 

Select appropriate prevention programs and 
practices to address these priority factors 
Combine programs and practices to ensure a 
comprehensive prevention approach 

Build a logic model for prevention with 


stakeholders 


Share and discuss the logic model 
with prevention partners, including 
those who will play a key role in 
selected programs and practices, to 
ensure that the model clearly 
communicates what they hope to 


accomplish and how 
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SPF Step 


Opportunities for Collaboration 


Example 


Step 4: 
Implementation: 
Communities deliver 
and fine-tune 
selected prevention 
programs and 


practices 


Adapt selected programs and practices as 
needed to increase their cultural relevance 
Establish infrastructure supports for the 
implementation of programs and practices 
Monitor programs and practices to ensure 


they are being implemented as intended 


Meet with leaders in 
implementation sites to work out 
the logistics of program/practice 
delivery, including identifying and 


training appropriate providers 


Step 5: Evaluation: 
Communities 
examine the process 
and outcomes of 
their programs and 


practices 


Identify evaluation stakeholders 

Plan and conduct a culturally appropriate 
and technically sound evaluation that will 
meet diverse stakeholder needs 
Determine how well each program and 
practice is working and why 

Share lessons learned and ensure use of 


evaluation findings 


Involve members of your focus 
population in the development of 


evaluation tools 


WANT TO LEARN MORE? 


Following are additional resources on the SPF and collaboration in prevention: 


e Applying the Strategic Prevention Framework (SPF): This section of the SAMHSA website offers 


detailed information on each SPF step and guiding principle, as well as many additional resources to 


guide and support effective prevention planning. 


e Grantee Stories: This section of the SAMHSA website includes stories of real-world experiences and 


strategies shared by recipients of SAMHSA grants and services. These stories illustrate how applying 


the SPF to prevention planning led to successful efforts in their states and communities. Many of these 


stories focus specifically on collaboration among diverse prevention stakeholders. 
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2. CRITERIA FOR SETTING PREVENTION PRIORITIES 


A comprehensive assessment of local substance use-related problems and contributing factors will typically 
yield information about more issues than a community can address at any given point in time. To focus and 
maximize the impact of local prevention efforts, planners need to make thoughtful and reasoned decisions 


about which problem(s) and factors to address first. Doing so requires clear prioritization criteria. 


IDENTIFYING THE PRIORITY PROBLEM 


The following criteria can help prevention planners examine and understand local assessment data with an eye 


toward identifying their community’s priority substance use-related problem: 


e Magnitude: This describes the prevalence of different problems. Prevention planners considering this 
criterion will try to answer the question: Which problem is affecting the greatest number of people in 


our community? 


e Severity: This describes how large an impact different problems are having on individuals and/or the 
community overall. Prevention planners considering this criterion will try to answer the question: 


Which problem is most serious? 


e Trend: This describes whether and how different problems seem to be changing over time within a 
community. Prevention planners considering this criterion will try to answer the question: Which 


problem, if any, is getting worse? 


e Changeability: This describes how likely it is that a community will be able to modify different 
problems. Prevention planners considering this criterion will try to answer the question: Which 


problem are we in the strongest position to favorably influence through prevention efforts? 


The most straightforward decision-making process is one in which careful consideration of all four criteria 
point to the same substance use-related problem—but this does not always happen. For example, prevention 
planners considering the severity of underage drinking as compared with youth prescription drug misuse in 
their community may find that alcohol is involved in more injuries, hospitalizations, and arrests among local 
youth than prescription drugs—but more young people may have died from overdosing on prescription drugs 
than from misusing alcohol. Prevention planners, together with key community stakeholders, must carefully 
consider and critically discuss their assessment data in relation to each of these criteria when weighing the 
relative burdens of different problems and when trying to understand their community’s capacity to reduce 


those burdens. 
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IDENTIFYING PRIORITY RISK AND PROTECTIVE FACTORS 


After identifying a priority problem, prevention planners will need to identify which associated risk and 
protective factors their community should address in order to influence that problem (see 3. Key Features of 
Risk and Protective Factors for more information). To identify priority risk and protective factors at the local 


level, it is helpful to consider their relative importance and changeability. 


e Importance: This describes the role of a specific risk or protective factor in reducing a problem. 
Prevention planners considering this criterion will try to answer the following questions: 
o To what degree is this factor contributing to our community’s priority problem? 
o Is this factor relevant to our focus population (e.g., based on developmental stage, culture)? 


0. Is this factor associated with other pressing behavioral health problems in our community? 


e Changeability: This describes a community’s capacity to influence a specific risk or protective factor. 
Prevention planners considering this criterion will try to answer the following questions: 
o Does our community have sufficient resources and readiness to address this factor? 
o Does asuitable prevention program or practice exist to address this factor? 


o Will our community be able to produce positive outcomes within a reasonable time frame? 


When setting local prevention priorities, it is best for a community to prioritize risk and protective factors that 
are high for both importance and changeability. |f no factors are high for both, the next best option is to 
prioritize factors with high importance and low changeability. Since factors with high importance contribute 
significantly to priority problems, addressing these factors is more likely to make a significant difference—and 
it is easier to increase the changeability of a factor (e.g., by boosting prevention capacity) than its importance. 
However, sometimes a community will choose to address a factor with low importance and high changeability. 
Doing so can give a community a quick “win,” help raise awareness of and support for prevention, and increase 


the community’s capacity to address more important factors in the future. 
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3. KEY FEATURES OF RISK AND PROTECTIVE FACTORS 


Since a community cannot change substance use-related problems directly, prevention practitioners must 
work through the underlying risk and protective factors that influence these problems. Risk factors precede, 
and are associated with a higher likelihood that a person will experience, a problem. Protective factors are 
associated with a lower likelihood that a person will experience a problem or that reduce the impact of one or 


more risk factors. Risk and protective factors: 


e Operate in multiple contexts 
e Are correlated and cumulative 
e Can be associated with multiple outcomes 


e Are influential across contexts and over time 
Each of these features is described below. 


RISK AND PROTECTIVE FACTORS OPERATE IN MULTIPLE CONTEXTS 


Many different factors can make a person vulnerable to, or resilient in the face of, potential substance use- 
related problems. At the individual level, these factors may include a person’s prenatal exposure to 
substances; genetic predisposition to addiction; substance-related knowledge, attitudes, and experiences; self- 


image; and social and emotional competence. 


But individuals do not exist in isolation; they are also influenced by numerous risk and protective factors 


operating at the relationship, community, and societal levels. For example: 


e At the relationship level, risk factors can include having parents and/or friends who use or misuse 
substances; protective factors can include having strong familial bonds and parents who disapprove of 


substance misuse and closely monitor youth behavior. 


e Atthe community level, risk factors can include social norms favorable to substance use or misuse and 
neighborhood poverty or violence; protective factors can include availability of after-school activities 


and health-related resources. 


e Atthe societal level, risk factors can include limited economic opportunity and substance use-related 
stigma; protective factors can include laws and policies that limit the number of alcohol retailers and 


prohibit sales to minors. 
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RISK AND PROTECTIVE FACTORS ARE CORRELATED AND CUMULATIVE 


Risk factors tend to be positively correlated with one another and negatively correlated with protective 
factors. In other words, a person who already has some risk factors is more likely than a person who does not 


have risk factors to experience even more risk factors—and less likely to experience protective factors. 


Risk and protective factors also tend to have a cumulative effect on the development, or prevention, of 
substance use and other behavioral health problems. Someone with multiple risk factors is more likely than 
someone with fewer risk factors to experience a problem; in contrast, someone with multiple protective 


factors will have greater protection against problems than someone with fewer protective factors. 


These correlations underscore the importance of intervening early and implementing prevention programs 


and practices that target multiple risk and protective factors. 


RISK AND PROTECTIVE FACTORS CAN BE ASSOCIATED WITH MULTIPLE OUTCOMES 


Although prevention programs and practices are often designed to produce a single outcome, both risk and 
protective factors can be associated with multiple outcomes. For example, adverse life experiences, such as 
abuse and homelessness, are associated with substance use-related problems as well as with anxiety, 
depression, and other behavioral health issues. Prevention efforts that target risk and protective factors 
common across multiple health-related problems have the potential to produce positive effects in many 


different areas of a person’s and community’s well-being. 


RISK AND PROTECTIVE FACTORS ARE INFLUENTIAL ACROSS CONTEXTS AND OVER TIME 


Risk and protective factors in one context, such as relationships, may also influence or be influenced by factors 
in another context, such as the community. For example, effective parenting has been shown to mediate the 
effects of multiple risk factors, including poverty. Risk and protective factors can also have power throughout a 
person’s entire life span. For example, risk factors such as early poverty and family dysfunction can contribute 


to the development of mental and/or substance use disorders later in life. 


WANT TO LEARN MORE? 


Following are additional resources on risk and protective factors associated with substance use-related 


problems: 


e Preventing Prescription Drug Misuse: Understanding Who Is at Risk: This tool provides information 
from cross-sectional and longitudinal studies on factors that have been shown to increase risk of or 


protect against the nonmedical use of prescription drugs. 
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e Preventing Youth Marijuana Use: Factors Associated with Use: This tool offers a summary of research 
findings on risk and protective factors associated with youth marijuana use. 


e Ensuring the Well-being of Boys and Young Men of Color: Factors that Promote Success and Protect 
Against Substance Use and Misuse: This tool distills research on substance use and misuse in boys and 
young men of color as well as on factors promoting positive youth development. 


e Risk and Protective Factors Associated with Binge or Heavy Episodic Drinking Among Adolescents 
and Young Adults: Using Prevention Research to Guide Prevention Practice: This tool provides an 
overview of prevention research on risk and protective factors associated with binge and heavy 


episodic drinking among adolescents and young adults. 
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4, LOGIC MODELS FOR PREVENTION 


Prevention planners use logic models to visualize how the pieces of a prevention plan fit together to produce 


change. As shown in the graphic below, key elements of a logic model typically include the following: 


e Priority problem a community has identified and intends to address through prevention efforts 

e Priority risk and protective factors a community intends to address in order to influence this problem 
e Prevention programs and practices selected to address these risk and protective factors 

e Short- and long-term outcomes a community expects to produce by implementing these programs 


and practices 


Priority Prevention 


Priority Risk and Programs 
Problem Protective and 
Factors Practices 


Like a road map, a logic model can help prevention planners and communities see where they are, where they 


are going, and how they will get there. 


WHY DEVELOP A LOGIC MODEL? 


Developing a logic model is a crucial step in strategic prevention planning that can help prevention planners 


and communities do the following: 


e Document key lessons learned and decisions made. A great deal of information-gathering and 
processing must occur before planners are ready to build a logic model of their community’s 
prevention plan. Logic models offer a concrete way to record, and create a snapshot of, key findings 


and decisions. 


e Identify any holes that need to be filled. The pieces of a logic model should fit together seamlessly. 
The act of building one can reveal any flaws in reasoning or planning gaps. For example, prevention 
planners might notice a priority factor with no program or practice to address it or an anticipated 
outcome with no program or practice to produce it. The sooner mistakes are discovered, the easier 


they are to correct. 


e Communicate how prevention efforts will work. A complete and hole-free logic model provides 


prevention planners with a valuable tool for clearly communicating with diverse stakeholders about 
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how their community intends to address its priority problem and the solid rationale behind this plan 


that makes it likely to succeed. 


e Assess and improve how well prevention efforts are working. By developing a logic model before 
implementing prevention programs and practices, planners build a solid foundation for evaluation. 
This model details the specific and measurable outcomes these prevention efforts are expected to 
produce in both the short and the long term, which helps planners and their evaluation colleagues 
monitor the progress of and continually improve these efforts—and, in the end, determine which 


efforts are worth continuing. 


LOGIC MODEL VARIATION 


There are many different ways to structure a logic model. For example, the logic model in this resource 
includes four elements, but others include a greater number of more detailed elements. Prevention planners 
should explore different approaches to building a logic model in order to identify the one that will best meet 
their needs. In some cases, they may want to build multiple logic models to serve different strategic planning 
purposes. For example, a detailed logic model may be needed to support evaluation efforts while a simple one 


may be more helpful when communicating with the public. 


WANT TO LEARN MORE? 


Following are additional resources on developing and using logic models in prevention: 


e CDC Evaluation Documents, Workbooks and Tools: Logic Models: This section of the Centers for 
Disease Control and Prevention’s (CDC) website offers links to several useful resources on 


understanding, developing, and using logic models to support the strategic planning and evaluation of 
health-related initiatives. 


e Developing a Logic Model to Guide Program Evaluation: This presentation from SAMHSA 
describes how logic models can be used to support prevention planning, implementation, and 


evaluation. 
e Examples of logic models from SAMHSA: 


o Examples of Community-level Logic Models for Reducing the Non-Medical Use of Opioid 
Prescription Drugs 


o Examples of Community- and State-level Logic Models for Addressing Opioid-related 
Overdose Deaths 


o Examples of Local-level Logic Models for Addressing Behavioral Health Disparities 
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5. CHARACTERISTICS OF A COMPREHENSIVE APPROACH 


When prevention programs and practices are planned and implemented in a strategic, coordinated, and 
comprehensive manner, they can support and reinforce one another and produce stronger health-related 
outcomes for individuals, families, and communities. In fact, the greatest evidence of effectiveness in 
addressing substance misuse and related behavioral health problems comes from studies of prevention 
programs and practices working in concert with one another—making a strong case for taking a 
comprehensive approach to prevention when and where possible. A comprehensive approach to the 
prevention of substance misuse and related behavioral health problems comprises multiple programs and 


practices that: 


e Address different levels of risk 
e Operate at different levels of influence 


e Require the support and participation of diverse stakeholders 


Each of these characteristics is described below. 


ADDRESSING DIFFERENT LEVELS OF RISK 


Community-based efforts to address substance use-related problems are most effective when matched to 
their audience’s level of risk. Prevention programs and practices can be grouped according to three levels of 


risk: universal, selective, and indicated. 


e Universal prevention efforts focus on general audiences who have not been identified based on 
substance use-related risk. 


e Selective prevention efforts focus on audiences with known risk factors for a substance use-related 
problem. 


e Indicated prevention efforts focus on audiences who are already experiencing a substance use-related 
problem. 


A community’s comprehensive approach to prevention should include strategies at all three levels. For 
example, a comprehensive plan for addressing the nonmedical use of prescription drugs among youth might 
include a school-based substance misuse prevention curriculum for all middle school students (universal), 
support groups for youth with a family history of substance misuse disorders (selective), and counseling and 


referral to other services, as needed, for youth who are currently misusing prescription drugs (indicated). 
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OPERATING AT DIFFERENT LEVELS OF INFLUENCE 


In addition to addressing different levels of risk, the programs and practices that make up a community’s 
comprehensive prevention approach should reduce the specific risk factors and strengthen the specific 
protective factors that are known to influence a substance use-related problem at the local level. Because 
these factors operate at different levels of a person’s experience (see 3. Key Features of Risk and Protective 
Factors for more information), so too should the programs and practices selected to address them. Specifically, 
a community’s comprehensive approach should include programs and practices that operate at the individual, 


relationship, and community levels. 


e To reduce risk at the individual level, programs and practices would address such factors as a person’s 
knowledge, attitudes, perceptions, and behaviors related to the substance use-related problem. For 
example, a school-based curriculum could boost participants’ knowledge of the risks and promote 


accurate perceptions of the harm associated with prescription drug misuse. 


e To reduce risk at the relationship level, programs and practices would address factors within a 
person’s closest social circle, such as the knowledge, attitudes, perceptions, and behaviors of family 
members, friends, and trusted service providers. For example, schools might supplement a prevention 
curriculum for students with a complementary program for parents that increases their understanding 
of the risks associated with prescription drug misuse and promotes effective communication with their 
children about these risks. 


e To reduce risk at the community level, programs and practices would address factors within a person’s 
broader social and physical environment, such as substance use-related social norms and access to 
substances. For example, a community might adopt prescriber and pharmacy guidelines designed to 


reduce medically inappropriate access to prescription drugs among youth. 


REQUIRING SUPPORT AND PARTCIPATION FROM DIVERSE STAKEHOLDERS 


A comprehensive approach to prevention is only possible with the support and participation of a broad range 
of community stakeholders. Stakeholders who can play an important role in addressing substance-use related 


problems include those with: 


e Access to and insights about relevant data—such as public health, health care, law enforcement, and 


education professionals 
e Expertise in gathering and using data—such as research and evaluation professionals 


e Capacity to raise awareness of prevention needs and build support for efforts to address them— 


such as media professionals and public opinion leaders 
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e Authority to allocate resources to prevention efforts—such as government officials, 


business/organizational leaders, and philanthropic organizations 


e Commitment and connections to focus population members—such as schools, workplaces, health 


clinics, community centers, non-profit agencies, and faith-based organizations 


Every community will have a unique comprehensive prevention approach because every community has a 
unique set of prevention priorities and resources. What will be similar across communities is the need for 
support and participation from diverse stakeholders to fully understand and establish those priorities, 
determine how best to address them, and leverage the resources required to do so successfully. In addition, 
the communication channels and working relationships fostered when diverse stakeholders collaborate to 
address current prevention needs also serve to strengthen their communities’ capacity to recognize and 


respond effectively as those needs evolve over time. 
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6. FINDING EVIDENCE-BASED PROGRAMS AND PRACTICES 


Following are resources that prevention planners can use to find evidence-based programs and practices 
(EBPPs) designed to help address substance misuse and related behavioral health problems in their 
communities. These resources from the Substance Abuse and Mental Health Services Administration 
(SAMHSA), other federal agencies, and nongovernmental organizations include searchable databases, reports, 


and sets of links to other sources of information. Please note that this is not an exhaustive list. 


RESOURCES FROM SAMHSA 


e Evidence-Based Practices Resource Center: This searchable database offers information about 
evidence-based practices related to substance use and mental health disorder prevention, treatment, 


and recovery. 


Finding Evidence-based Programs: This resources contains links to numerous federal and 
nongovernmental sources of information on evidence-based programs and practices in substance 


misuse prevention and behavioral health promotion. 


RESOURCES FROM OTHER FEDERAL AGENCIES 


CrimeSolutions.gov: This searchable database from the National Institute of Justice offers information 
about the strength of evidence supporting the effectiveness of programs and practices in such areas as 


substance abuse, juvenile delinquency, and crime prevention. 


Evidence-Based Practices & Programs: This section of the National Institutes of Health’s website 
includes information from federal agencies, nonprofit organizations, and private-sector organizations 


on evidence-based public health services, programs, and practices. 


The Community Guide: This online guide includes evidence-based findings and recommendations of 
the Community Preventive Services Task Force—an independent group established in 1996 by the U.S. 
Department of Health and Human Services comprising public health and prevention experts appointed 
by the director of the Centers for Disease Control and Prevention. Topics include adolescent health, 


alcohol, and mental health. 


° Healthy People 2020 Evidence-Based Resources: This searchable database from the U.S. Department 
of Health and Human Services offers resources on various topics related to Healthy People’s science- 
based, 10-year national objectives for improving health. Topics include access to health services, 


educational and community-based programs, and substance abuse. 
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Model Programs Guide: This searchable database from the Office of Juvenile Justice and Delinquency 
Prevention contains information about evidence-based juvenile justice and youth prevention, 


intervention, and reentry programs. 


youth.gov: Created by the Interagency Working Group on Youth Programs, this website includes a 
searchable program directory of evidence-based programs to address delinquency and other problem 
behaviors in young people as well as the Evidence-Based Program Directories, a set of links to other 
federal resources. 


RESOURCES FROM NONGOVERNMENTAL ORGANIZATIONS 


e Blueprints for Healthy Youth Development: This searchable database from the Center for the Study 
and Prevention of Violence at the University of Colorado Boulder offers information about evidence- 
based positive youth development programs, including those to prevent substance use and misuse and 


to promote positive relationships and academic achievement. 


e Resources and Programs: This searchable database from the Suicide Prevention Resource Center 


offers information on evidence-based programs, practices, and strategic planning. 


e CASEL Program Guides: Effective Social and Emotional Learning Programs: These resources from the 
Collaborative for Academic, Social, and Emotional Learning (CASEL) offer findings from systematic 


reviews of school-based (preschool through high school) social and emotional learning programs. 


e Top Tier Evidence: This resource from the Laura and John Arnold Foundation’s Evidence-Based Policy 
Initiative offers findings from a systematic review of interventions in such areas as early childhood, 
education (K-12), youth development, crime/violence prevention, substance abuse prevention and 


treatment, and housing/homelessness. 


e What Works: This searchable database from Child Trends offers information on programs that 
promote outcomes related to education, life skills, and social/emotional, mental, physical, or 
reproductive health. 
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7. TYPES OF EVALUATION 


Evaluation is the systematic collection and analysis of information about prevention efforts to reduce 
uncertainty, improve effectiveness, and make decisions. According to Michael Quinn Patton (2002), an expert 
in evaluation, research seeks to prove while evaluation seeks to improve. In other words, evaluation is all about 
enhancing prevention efforts. Two important types of evaluation in prevention are process evaluation and 


outcome evaluation. 


PROCESS EVALUATION 


Process evaluation documents the implementation of prevention programs and practices. This type of 
evaluation can be used to continually improve prevention efforts as they are occurring and to enhance 
stakeholder understanding of the outcomes ultimately produced by these efforts. Specifically, process 


evaluation can help communities answer the following questions: 


e To what extent was the program or practice implemented as originally designed? 
e What, if any, adaptations were made to the program or practice? 
e How many people participated in the program or were affected by the practice? 


e Were these participants the intended audience? 


Approaches to gathering data to answer these and other process evaluation questions include interviews and 
focus groups with providers and/or participants, attendance records, feedback forms, photography and video, 


implementation checklists, and journal entries/field notes. 


OUTCOME EVALUATION 


Outcome evaluation measures the effects of prevention programs and practices following their 
implementation. This type of evaluation can be used to determine whether, and to what degree, prevention 
efforts are producing the short- and long-term outcomes a community wants to bring about. Specifically, 


outcome evaluation can help communities answer the following questions: 


e Did any changes occur among participants and/or within the community? 
e If changes did occur, what were they? 
e Can these changes be attributed to the program and/or practice itself? 


e How do these changes compare to those anticipated by the community? 


Because outcome evaluation seeks to capture and understand any changes that occur from before to after the 
implementation of prevention programs and practices, it is important to gather data from participants at both 
points in time. Two approaches to designing outcome evaluation studies to gather these data are experimental 


and quasi-experimental. 
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e Experimental evaluation design: In this design, which offers the highest level of scientific rigor, 
participants are randomly assigned to a program/practice group or to a control group. Those in the 
program/practice group participate in the intervention being evaluated while those in the control 
group do not. Data are gathered from both groups before and after the intervention using such 
methods as surveys, interviews, focus groups, and/or observations. Results are then compared to 


identify any differences between groups. 


e Quasi-experimental evaluation design: This design is similar to the experimental design in that there is 
a program/practice group and a control group; however, it does not rely on random assignment. Pure 
randomization is easier to achieve in laboratories where potentially influential variables can be 
controlled than in the complex, real-world settings where prevention efforts typically occur. In quasi- 
experimental evaluations of prevention efforts, existing groups are used rather than randomly 
assigning individuals to groups. Because there is no random assignment, the program/practice group 
and the control group will differ not only in terms of the intervention experience but also in other 
potentially important ways. Evaluators using quasi-experimental designs must try to identify, and 
explore the potential impact of, as many of these between-group differences as possible to 
understand their results—including whether the intervention or other variables can be considered 


responsible. 


If a community can credibly demonstrate that existing prevention programs and practices are working well and 
favorably influencing community health and well-being, this can go a long way toward building confidence in 


and ongoing support for these—and future—prevention efforts. 


WANT TO LEARN MORE? 


Following are additional resources on evaluating prevention programs and practices: 


e Evaluation Organizations: This section of the SAMHSA website offers a list of federal and 
nongovernmental organizations (NGOs) that provide evaluation-related training, technical assistance, 


and resources for prevention practitioners. 


e Evaluation Tools and Resources: This section of the SAMHSA website offers a set of practical tools, tip 
sheets, and other resources from federal organizations and NGOs with evaluation and prevention 


expertise. 


e Step 5: Evaluate: This section of the SAMHSA website is dedicated to the fifth step of the Strategic 
Prevention Framework (SPF) and includes detailed information on such topics as evaluating prevention 


processes and outcomes and communicating evaluation results. 
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